6 Concluding Remarks 5.2 Incorporate Database Categorization 5 Improving the Retrieval of Relevant Documents = 2:0m 2 2 2 2 2 2 2 2 2 2 2 = 2:0m 2 2 2 2 2 2 2 2 2 2 2

In this paper, we proposed a new, highly scalable method to solve the database selection problem when the number of local databases in a metasearch engine is very large. For single-term queries, this method guarantees all desired documents to be retrieved. Experiments were conducted to show that this method is very eeective for typical short Internet queries. We also illustrated that the our database selection and collection fusion method can be incorporated with other techniques to improve the retrieval eeectiveness. We plan to continue this research in the following directions. (1) Further improve the accuracy of our method. Study in 38] indicates by incorporating certain dependencies among adjacent query terms, the accuracy can be further improved. One research issue is how to incorporate such dependencies in the integrated database representative. (2) Extend our method that works very well for short Internet queries so that it can also work well for longer queries. a given query q if and only if MDR(q; D 1) > MDR(q; D 2) > ::: > MDR(q; D N), where MDR(q; D i) is the degree of relevance of the most relevant document in database D i with the query q. The proof of Proposition 3 and the description of a method for estimating the degree of relevance of the most relevant document in a database can be found in 41] and will not be repeated here. It was reported in 41] that the method yields retrieval eeectiveness which is very close to that as if all documents were placed in one site (cor iden doc ranges from 88% to 98%.) As mentioned earlier, one reason for Internet queries to have low retrieval eeectiveness is because Internet queries are usually very short and short queries do not provide enough context words to determine the meanings of their terms. One technique to address this problem is to rst assign queries and databases to appropriate topics or concepts and then limit the search of databases, with respect to a given query, to only those databases that share the same concepts as those assigned to the query. The idea is to utilize these concepts as context to help disambiguate the meanings of terms in databases and queries, and as a result, leading to the improvement of retrieving relevant documents. Several researchers have studied the idea of assigning databases to concepts and/or clustering documents into new databases to improve retrieval …